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Abstract 


The  numerous  reasons  for  teaching  thinking  have  been  tied  to  the  perceived  needs  of  the 
present  and  future  technological  society.  Expectations  are  placed  on  individuals  to  be 
self-sufficient,  responsible  citizens,  and  part  of  an  efficient  work  force,  oriented 
toward  an  information  age.  Endorsement  of  this  objective  and  attempts  to  teach  thinking 
have  resulted  in  the  proliferation  of  many  learning/thinking  programs.  Few  studies  to 
date  have,  however,  systematically  and  longitudinally  evaluated  any  of  the  available 
programs  to  determine  which  ones  are  most  effective  and  most  easily  integrated  into  the 
regular  school  curriculum.  More  importantly,  there  is  little  empirical  data  comparing 
the  efficacy  of  the  programs  at  different  grade  levels  or  with  different  types  of  learners. 

The  purpose  of  the  Cognitive  Education  Project,  centered  at  the  University  of 
Alberta,  was  to  undertake  a three-year  longitudinal  evaluation  of  two  cognitive  education 
programs.  The  critical  difference  between  the  two  experimental  programs  was  that  one 
(Feuerstein's  Instrumental  Enrichment  - I.E.)  was  taught  out  of  curricular  content, 
while  the  second  program  (Strategies  Program  for  Effective  Learning/Thinking  - 
S.P.E.L.T.)  was  taught  directly  within  curricular  content. 

Specifically,  the  effectiveness  of  the  I.E.  and  S.P.E.L.T.  cognitive  education 
programs  was  compared  with  traditional  instruction  at  two  initial  grade  levels  (grades  4 
and  7)  for  three  diagnostic  groups  (gifted,  learning  disabled  and  normal  achievers).  The 
comparison  was  done  in  terms  of: 

a.  the  effects  of  the  programs  on  students'  affect  and  motivation,  academic 
achievement,  cognitive  ability,  and  learning/thinking  and  problem  solving 
strategies; 

b.  the  differential  impact  of  the  programs; 

c.  the  feasibility  of  implementing  learning/thinking  strategies  instructional 
programs  as  part  of  the  regular  curriculum  of  schools;  and 

d.  identifying  appropriate  methods  for  providing  the  level  and  quality  of 
teacher  training  necessary  for  implementation. 

These  general  objectives  gave  rise  to  a number  of  questions  spanning  student, 
teacher,  parent  and  administrator  responses.  The  study  was  implemented  in  two  phases 
starting  in  1984  and  1985  respectively  and  overlapping,  with  phase  1 ending  in  1987 
and  phase  2,  in  1988.  It  utilized  a repeated  measures  factorial  design  involving  three 
types  of  instructional  programs,  three  categories  of  students,  and  two  initial  grade 
levels  (grades  4 and  7).  The  complete  study  provided  four  data  points. 
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In  the  course  of  the  implementation,  teachers  of  the  control  condition  taught  as 
usual,  whereas  teachers  assigned  to  the  two  cognitive  education  procedures  received 
intensive  inservice  training  prior  to  classroom  strategy  instruction. 

In  identifying  subjects  for  the  study,  intellectual,  academic  and  behavioral 
characteristics  were  used,  resulting  in  the  selection  of  900  students  from  an  initial 
population  of  4,000.  Based  upon  intelligence  test  scores  and  achievement  three  groups 
of  subjects  were  identified  as  gifted,  average  and  learning  disabled. 

Apart  from  obtaining  responses  from  students,  questionnaires  were  administered 
to  teachers,  parents  and  administrators  to  assess  their  perceptions  of  the  different 
programs.  The  results  of  the  study  indicate  that: 

1.  Cognitive  education  was  effective  in  improving  student  thinking,  especially 
for  the  grade  4 learning  disabled,  and  to  a lesser  extent  the  gifted,  in  reading 
comprehension,  and  comprehension  monitoring  skills.  Students'  strategic  behavior 
generally  improved  across  grade  and  diagnostic  groups. 

2.  Though  I.E.  was  effective,  S.P.E.L.T.  tended  to  produce  more  positive  changes 
in  students'  overall  performance. 

3.  Teachers',  parents',  and  administrators'  responses  were  positive  towards  the 
two  cognitive  education  programs,  with  more  favorable  remarks  being  made  for 
S.P.E.L.T.  The  teachers  involved  in  the  study  indicated  general  appropriateness  of  both 
experimental  programs  especially  for  grade  4 students.  The  vast  majority  of 
experimental  teachers  said  the  inservices  provided  were  appropriate  and  that  they  would 
continue  to  use  the  instructional  procedures  from  their  respective  program.  As  well, 
the  teachers  said  they  would  recommend  the  two  programs  to  their  colleagues.  Parents 
also  indicated  observation  of  positive  changes  in  their  youngster's  self-confidence,  task 
persistence,  accepting  alternative  points  of  view,  originality  of  thinking  and  ques- 
tioning, etc. 

4.  The  question  of  whether  either  of  the  cognitive  education  programs  was  more 
effective  for  specific  groups  of  children  did  not  receive  a definitive  answer  and  may  have 
to  be  further  examined.  On  the  whole,  the  experimental  programs  appeared  to  be  most 
effective  for  learning  disabled  students,  and  to  a lesser  extent,  the  gifted  at  both  grade 
levels., 
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The  Cognitive  Education  Project 
Summary  Report 


The  purpose  of  this  report  is  to  summarize  the  major  features  and  findings  of  the 
Cognitive  Education  Project.  First,  the  rationale,  objectives  and  research  questions  of 
the  project  are  outlined.  These  are  followed  by  a brief  description  of  the  two  cognitive 
education  programs  (Instrumental  Enrichment  and  Strategies  Program  for  Effective 
Learning/Thinking)  selected  for  comparison  in  the  project.  Research  methodology  and 
findings  are  then  presented.  Finally,  conclusions  and  recommendations  are  specified. 


Rationale 

Over  the  past  decade,  there  has  been  a significant  increase  in  interest  and  desire  from 
teachers,  school  districts,  colleges,  and  universities  to  teach  thinking  skills.  This 
interest  has  been  associated  with  the  emergence  of  various  cognitive  education  programs 
aimed  at  enhancing  students'  cognitive  and  metacognitive  skills.  This  has  been  done  with 
the  hope  that  students  might  become  more  independent  learners  and  more  efficient 
problem  solvers. 

Although  cognitive  education  programs  have  been  growing  in  number  and 
popularity,  many  questions  concerning  their  effectiveness  remain.  Part  of  the  reason  is 
the  lack  of  long-term  evaluation  of  these  programs.  First,  the  packages  available  today 
tend  to  vary  in  terms  of  scope,  skill  development,  age/grade  suitability,  training 
requirements,  curriculum  integration,  cost,  and  instructional  methodology.  Few  studies 
to  date  have  systematically  evaluated  any  of  the  available  programs  to  determine  which 
ones  are  most  effective  and  most  easily  integrated  within  the  regular  school  curriculum. 
More  importantly,  there  is  little  empirical  work  comparing  the  efficacy  of  the  programs 
at  different  grade  levels  for  different  types  of  learners.  The  results  of  these  studies  are 
often  equivocal  with  regard  to  program  effectiveness. 

Associated  with  this  lack  of  comprehensive  evaluation  is  the  question  of  selecting 
from  alternative  approaches  to  teach  thinking.  That  is,  linking  this  evaluation  issue  to 
instruction,  the  central  question  becomes  how  to  teach  learning/thinking.  Educators  are 
faced  with  choosing  between  an  in-content  or  out-of-content  instructional 
approach.  Which  would  work  better  for  students?  Would  there  be  differences  if  we 
compare  these  two  approaches  at  different  grade  levels  for  different  types  of  learners? 
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Basing  our  conclusions  and  subsequent  action  on  subjective  preferences  alone  would  be 
inadequate.  The  seeming  course  is  to  conduct  an  evaluation  study:  putting  in-content  and 
out-of-content  programs  in  place  in  selected  schools  and  evaluating  the  results.  With 
this  main  purpose,  the  Cognitive  Education  Project  (CEP)  was  inaugurated  in  the  fall  of 
1984. 


Objectives  and  Research  Questions 

The  Cognitive  Education  Project  was  a cooperative  venture  involving:  1)  Alberta 
Education,  Government  of  Alberta;  2)  the  Department  of  Educational  Psychology,  the 
University  of  Alberta;  and  3)  various  school  jurisdictions  in  north-central  Alberta.  It 
was  established  with  the  general  purpose  of  undertaking  a long-term  evaluation  of  two 
cognitive  education  programs  (out-of-content  versus  in-content)  in  relation  to 
traditional  instruction  in  elementary  and  junior  high  classrooms. 

For  the  out-of-content  approach,  Feuerstein's  Instrumental  Enrichment  (I.E.) 
was  selected  because  it  is  one  of  the  most  comprehensive  and  field-tested  learning  and 
thinking  programs  available  to  date.  It  represents  an  out-of-content  instructional 
approach  utilizing  paper-pencil  tasks  and  intensive  teacher-pupil  discussion  to  teach 
learning/thinking  skills.  In  contrast,  the  Strategies  Program  for  Effective 
Learning/Thinking  (S.P.E.L.T.)  was  chosen  as  an  in-content  instructional  approach.  It 
integrates  the  features  of  several  prominent  cognitive  theorists  and  intervention 
procedures,  and  it  emphasizes  the  teaching  of  learning/thinking  strategies  directly 
within  content  across  the  curriculum. 

The  effectiveness  of  cognitive  education  — represented  by  the  I.E.  and  S.P.E.L.T. 
programs  — was  compared  with  the  effects  of  traditional  instruction  at  two  initial  grade 
levels  (grades  4 and  7)  for  three  diagnostic  groups  (gifted,  learning  disabled  and 
average  achievers).  Specifically,  the  objectives  of  the  project  were  fourfold: 

1 . to  assess  the  relative  effectiveness  of  the  two  programs  in  terms  of  their  impact  on 
students'  affect  and  motivation,  academic  achievement,  cognitive  ability,  and 
learning/thinking  and  problem-solving  strategies; 

2.  to  examine  the  differential  effects  of  the  programs  on  gifted,  normal  achieving,  and 
learning  disabled  students; 

3.  to  ascertain  the  feasibility  of  implementing  learning/thinking  strategies 
instructional  programs  on  a large  scale  as  part  of  the  regular  curriculum  of 
schools;  and 
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4.  to  identify  appropriate  methods  for  providing  the  level  and  quality  of  teacher 
training  necessary  for  implementation. 


In  response  to  the  objectives  listed  above,  nine  research  questions  were  formulated. 


1 . What  are  the  relative  effects  of  the  different  cognitive  education  programs  compared 
to  traditional  instruction  on  gifted,  learning  disabled,  and  average  achievers  with 
respect  to  the  following  dimensions? 

a perceived  competence 

b.  perceived  locus  of  control 

c.  performance  in  reading  comprehension 

d.  performance  in  arithmetic  problem  solving 

e.  use  of  specific  strategies  employed  in  solving  problems 

2.  Is/are  the  training  program(s)  more  appropriate  at  different  ages  for  different 
diagnostic  groups? 

3.  Do  the  pupils  continue  to  maintain  and/or  alter  their  level  of  performance  following 
the  withdrawal  of  training? 

4 . What  is  the  nature  of  strategy  monitoring  for  each  of  the  three  groups  across  the 
different  age/grade  groupings?  To  what  extent  can  the  cognitive  education  programs 
be  implemented  as  intended  by  program  developers? 

5 What  is  the  nature  of  the  strategies  utilized  by  each  of  the  groups  across  the 
different  age/grade  groupings  prior  to  instruction  and  at  the  conclusion  of  the 
study? 

6.  What  are  parents',  teachers',  and  administrators'  opinions  regarding  the  cognitive 
education  programs? 

7.  What  are  the  teachers'  and  administrators'  opinions  regarding  inservice  and 
consultative  assistance  provided  for  cognitive  education  programs? 

8.  What  guidelines  for  preservice  and  inservice  programs  for  teachers  seem 
appropriate? 

9.  How  well  are  the  programs  implemented  by  teachers  of  cognitive  education?  Do 
teachers  learn  and  implement  the  cognitive  education  strategies?  How  appropriate 
and  effective  are  the  inservice  and  consultation  provided? 
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The  Two  Selected  Programs  — Instrumental  Enrichment  and 
Strategies  Program  for  Effective  Learning/Thinking 

To  clarify  the  characteristics  of  experimental  treatments  in  the  research  methodology, 
the  two  cognitive  education  programs  selected  for  comparison  in  the  study  are  briefly 
described  here. 

Instrumental  Enrichment 

Feuerstein's  Instrumental  Enrichment  (I.E.)  (Feuerstein,  Rand,  Hoffman,  & Miller, 
1980)  was  selected  to  represent  the  out-of-content  approach  to  instruction.  This 
program  was  originally  designed  for  culturally  disadvantaged  children  and  youth. 
However,  it  is  currently  being  used  with  a broader  population  of  children  in  upper 
elementary,  junior,  and  senior  high  schools.  A distinguishing  feature  of  I.E.  is  its 
emphasis  on  the  importance  of  mediation  for  strategy  development.  Strategies  are  ways 
to  facilitate  the  acquisition,  manipulation,  integration,  storage  and  retrieval  of 
information  across  situations  and  settings.  For  Feuerstein,  training  for  such  strategy 
development  is  through  a "mediated  learning  experience".  This  means  that  an  adult 
mediator  elicits  behaviors  from  the  child  that  lead  to  the  solving  of  the  problem,  and 
then  the  child  comes  to  understand  the  goals  and  strategies  of  the  task.  Thus  in  I.E., 
social  interaction  is  important,  because  it  is  believed  that  it  is  not  the  content,  but  the 
means  of  interacting  that  is  internalized  by  the  child. 

The  Feuerstein  program  utilizes  pencil-and-paper  tasks  with  related  intensive 
teacher-student  discussion.  It  consists  of  15  'instruments'  or  pencil-and-paper 
exercises  as  follows:  Organization  of  Dots;  Analytic  Perception;  Orientation  in  Space  I, 

II,  III;  Comparisons;  Categorization;  Instructions;  Family  Relations;  Illustrations; 
Numerical  Progressions;  Temporal  Relations;  Stencil  Design;  Transitive  Relations;  and, 
Syllogisms.  While  the  names  of  the  individual  instruments  indicate  the  dimensions  of 
the  program,  the  program  is  intended  to  be  content-free.  The  term  content-free  is 
intended  to  convey  the  idea  that  the  contents  of  any  particular  exercise  are  merely  a 
vehicle,  or  an  instrument,  to  achieve  the  overall  goals  of  the  program.  The  major  goal  of 
I.E.  is  to  enhance  the  cognitive  modifiability,  that  is,  learning  potential,  of  the 
individual. 

The  I.E.  program  can  be  integrated  into  the  regular  school  curriculum,  but  is 
taught  in  a decontextualized  form  first.  Typically,  I.E.  instruction  extends  over  a two- 
to  three-year  period,  with  a minimum  of  three  sessions  per  week  devoted  to  work  on  the 
instruments.  An  I.E.  lesson  normally  begins  with  a ten-minute  introduction,  followed  by 
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individual  work,  class  discussion  and  summary.  The  teacher  ensures  that  adequate 
mediational  experiences  are  provided  to  students  especially  in  the  introductory  and 
discussion  stages.  A typical  lesson  in  I.E.  appears  to  be  highly  perceptual  with  strong 
visual-motor  factors;  however,  in  reality,  the  product  resulting  from,  say,  the 
connecting  of  dots  to  form  a pattern  in  the  Organization  of  Dots  instrument,  is  a very 
minor  part  of  I.E.  The  program  is  characterized  by  students'  discovering  a pattern  in  the 
instruments  through  mediation;  determining  the  underlying  principle,  then  "bridging" 
this  principle  to  other  examples.  It  is  this  dynamic  involvement  of  the  teacher  in  a 
dialogue  with  the  student,  along  with  the  change  in  orientation  from  product  to  process, 
that  depicts  this  program. 

Strategies  Program  for  Effective  Learning/Thinking 

The  Strategies  Program  for  Effective  Learning/Thinking  (S.P.E.L.T.)  (Mulcahy,  Marfo, 

& Peat,  1984)  was  selected  as  the  in-content  program  in  the  study.  It  was  initially 
aimed  at  children  in  the  upper  elementary  and  junior  high  school  grades.  It  has  since 
been  extended  to  high  school  and  college  populations.  Similar  to  the  I.E.  program,  the 
S.P.E.L.T.  approach  also  aims  to  foster  strategic  learning  and  emphasizes  the  importance 
of  the  interactions  between  the  teacher  and  students  in  the  process  of  strategy 
development.  In  the  S.P.E.L.T.  program,  cognitive  strategies  are  conceptualized  as 
internally  organized  skills  or  control  processes  by  which  learners  regulate  their 
cognitive  and/or  learning  behavior.  A learner's  repertoire  of  strategies  is  thus  seen  as  a 
set  of  tools  that  enables  him/her  to  more  effectively  and  efficiently  activate  and  regulate 
important  cognitive  activities  such  as  attention,  comprehension,  retention  and  retrieval 
of  information,  thinking,  and  problem  solving.  The  teacher  plays  the  role  of  a mediator 
between  the  learner  and  the  external  world,  structuring  the  learning  environment  and 
providing  opportunities  necessary  to  establish  and  improve  strategic  behavior  in 
learning,  thinking,  and  problem-solving  situations.  A general  teaching  orientation  is 
embedded  within  S.P.E.L.T.  whereby  the  teacher's  goal  in  all  planning  and  instruction  is 
to  actively  involve  the  student  in  the  learning  process  (Peat,  Mulcahy,  & Darko-Yeboah, 
1989). 

S.P.E.L.T.  however,  differs  from  I.E.  in  its  instructional  context.  While  I.E.  has 
been  designed  as  a structured  package  to  be  taught  independently  of  existing  curriculum 
content,  the  S.P.E.L.T.  approach  holds  that  the  teaching  of  learning/thinking  strategies 
should  take  place  within  content  and  not  as  an  independent  or  isolated  curricular 
activity.  With  regular  school  curriculum  as  a vehicle,  S.P.E.L.T.  through  inservice 
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demonstrates  how  the  teacher  can  use  specific  strategies  to  activate  and  regulate 
students'  learning  activities. 

Moreover,  S.P.E.L.T.  utilizes  a three-phase  instructional  perspective.  It 
progresses  from  the  first  phase  — direct  teaching  of  strategies  in  content  areas  — to  the 
second  phase  — teaching  for  strategy  transfer  — and  finally,  to  the  third  phase  where 
students  can  themselves  generate  new  strategies  to  acquire,  analyze,  and  apply 
information  and  ideas.  The  active  involvement  of  the  students  in  the  learning  process  is 
maintained  as  a goal  throughout  the  three  phases  of  S.P.E.L.T.  instruction.  For  both 
Phases  II  and  III,  the  teacher  engages  in  Socratic  Dialogue  — an  interactive  relationship 
between  the  teacher  and  students,  where  the  teacher  leads  the  students  through 
questioning  to  discover  relationships  for  themselves.  Thus,  mediational  teaching  is  also 
a feature  of  the  S.P.E.L.T.  program. 

In  sum,  the  common  goal  of  the  two  programs  is  to  help  students  learn  'how  to 
learn'  and  thus  become  independent,  organized,  active,  and  purposeful  thinkers  and 
problem  solvers.  The  critical  factor  for  distinguishing  between  the  two  is  the  nature  of 
integration  of  the  program  into  the  curriculum.  I.E.  is  considered  a detached  program 
because  it  is  first  taught  without  using  curriculum  content  and  later  integrated  into  the 
curriculum.  On  the  other  hand,  S.P.E.L.T.  is  described  as  an  embedded  program  because  it 
is  taught  directly  using  curriculum  content. 

To  evaluate  the  effectiveness  of  the  out-of-content  approach  of  I.E.  versus  the  in- 
content approach  of  S.P.E.L.T.  in  relation  to  conventional  instruction,  the  following 
research  methodology  was  developed. 


Methodology 

Research  design 

The  study  was  a three-year  longitudinal  evaluation  study  implemented  in  two  phases 
(i.e.,  phase  1:  1984  - 87;  and  phase  2:  1985  - 88)1.  It  utilized  a repeated  measures 
factorial  design  involving  the  three  types  of  instructional  programs  (I.E.,  S.P.E.L.T.,  and 
Control),  three  categories  of  students  (gifted,  average,  and  learning  disabled),  and  two 


1 Due  to  budget  restraints  the  study  was  conducted  in  two  major  phases.  Phase  I (1 984  - 

1987)  began  in  October  1984  and  ended  in  June  1987.  Phase  II  (1985  - 1988)  started 
in  October  1985  and  ended  in  June  1988.  Thus,  for  each  phase,  it  was  a three-year 
period  of  evaluation.  Nevertheless,  for  the  whole  study,  it  took  four  years  (1984  - 

1988)  to  complete. 
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Initial  grade  levels  (grade  4 and  grade  7).  The  complete  study  involved  four  data  points 
(repeated  measures):  pre-test  in  the  fall  of  the  initial  year,  and  two  post-tests  in 
succeeding  May/June  periods  corresponding  to  the  end  of  grades  4 , 5,  7,  and  8,  and  a 
maintenance  post-test  at  the  end  of  grades  6 and  9 (see  Table  1). 


Table  1 

Overall  Study  Design 


Oct.  84 

June  85 

Oct.  85 

June  86 

June  87 

June  88 

Phase  1: 

Grade  4 

All* 

PRE 

PT 1 

- 

PT2 

Maintenance 

- 

Grade  7 

I.E. 

PRE 

PT1 



PT2 

Maintenance 



axrrFO. 

PRE 

PT  1 

PT2 

Maintenance 

Phase  2: 

Grade  4 

I.E. 

PRE 

PT1 

PT2 

Maintenance 

S.P.E.L.T 

PRE 

PT1 

PT2 

Maintenance 

Grade  7 

All* 

PRE 

PT1 

PT2 

Maintenance 

* All  refers  to  all  3 instructional  conditions. 
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Program  implementation 

Teachers  assigned  to  the  control  condition  (traditional  instruction)  were  told  to  teach  as 
usual,  whereas  teachers  assigned  to  the  two  cognitive  education  procedures  received 
inservice  training  from  project  staff  prior  to  giving  strategy  instruction.  Thus  students 
in  the  control  condition  received  traditional  instruction,  while  students  involved  in  the 
two  cognitive  education  programs  received  a minimum  of  120  minutes  of  strategy 
instruction  per  week  over  two  school  years.  Strategy  instruction  was  followed  by  one 
year  of  maintenance,  during  which  all  strategy  instruction  was  withdrawn.  Since  I.E.  is 
an  out-of-content  program,  teachers  were  required  to  take  time  out  of  a variety  of 
curricular  content  areas  to  implement  the  program.  Essentially,  the  I.E.  instruction 
time  was  taken  from  language  arts.  For  S.P.E.L.T.,  teachers  incorporated  strategy 
instruction  across  content  areas,  and  language  arts  was  the  major  content  medium  for 
the  S.P.E.L.T.  instruction. 

Subject  identification 

Based  on  intellectual,  academic  and  behavioral  characteristics,  about  900  pupils 
comprising  gifted,  average,  and  learning  disabled  students  were  identified  from  the  total 
initial  population  of  2,400  students  in  1984-85  and  1,600  students  in  1985-86. 
Selection  criteria  were  as  follows: 

The  gifted  students  selected  for  this  project  were  those  who: 

• obtained  scores  of  115  or  higher  on  the  verbal  and  the  non-verbal  sub-scales 
of  the  Canadian  Cognitive  Abilities  Test  (CCAT), 

• were  rated  as  being  above  average  in  achievement  in  reading  and  at/or  above 
grade  level  in  math  on  the  Canadian  Achievement  Test  (CAT), 

• were  rated  as  being  above  the  mean  (of  the  total  study  population)  on  all 
three  of  the  Renzulli  and  Hartman  Scales  for  the  Rating  of  Behavioral 
Characteristics  of  Superior  Students  (SRBCSS)  categories  (i.e.,  motivation, 
learning  and  creativity  characteristics). 

The  average-achieving  students  included  in  this  study  were  those  who: 

• obtained  scores  within  one  standard  deviation  of  the  mean  on  both  the  verbal 
and  nonverbal  sub-scales  of  the  CCAT  (85-115), 

• obtained  achievement  scores  on  the  reading  and  math  sub-scales  of  the  CAT 
within  approximately  one  standard  deviation  of  the  mean. 

The  learning  disabled  students  identified  in  this  study  were  those  who: 

• obtained  scores  within  one  standard  deviation  of  the  mean  on  both  the  verbal 
and  nonverbal  sub-scales  of  the  CCAT  (85  - 115), 
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• obtained  achievement  scores  of  approximately  one  standard  deviation  or  more 
below  the  mean  on  the  reading  sub-scale  of  the  CAT. 

It  can  be  seen  that  in  the  subject  identification  process,  reading  was  chosen  as  the 
major  academic  measure  because  it  is  one  of  the  most  important  skills  necessary  for 
school  success.  Furthermore,  the  majority  of  learning  disabled  children  experience 
learning  difficulties  in  this  area.  Consequently,  it  was  the  critical  achievement  measure 
used  in  the  identification  of  all  three  diagnostic  groups  (i.e.,  gifted,  average  and  learning 
disabled). 

Assessment  of  program  effects 

The  instruments  utilized  in  this  study  can  be  grouped  into  four  categories:  (1)  cognitive 
ability,  (2)  academic  achievement,  (3)  affective  perceptions,  and  (4)  cognitive 
strategies. 

To  assess  general  inteiiectual/cognitive  ability,  the  Canadian  Cognitive 
Abilities  Test  (CCAT)  was  administered.  This  test  measures  verbal,  quantitative  and 
nonverbal  reasoning  abilities. 

For  academic  achievement,  the  Canadian  Achievement  Test  (CAT)  was 
utilized.  This  test  consists  of  two  separate  batteries  measuring  skills  in  reading 
(vocabulary,  comprehension)  and  mathematics  (computation,  concepts,  application). 

To  assess  affective  perceptions,  several  measures  were  employed: 

a.  Perceived  competence : Harter's  Perceived  Competence  Scale  was  used  to 
measure  students'  self-perceptions  in  four  areas  (i.e.,  cognitive,  social, 
physical  and  general). 

b.  Self-concept  Coopersmith's  Self  Esteem  Inventories  were  employed  to 
assess  evaluative  attitudes  toward  the  self  in  social,  academic,  family  and 
personal  areas  of  experience. 

c.  Locus  of  control : Crandall's  Intellectual  Achievement  Responsibility 
Questionnaire  (IARQ)  was  administered  to  examine  students'  beliefs 
regarding  responsibility  for  outcomes  in  academic  achievement 
situations. 

To  measure  cognitive  strategies,  several  tests  were  selected  or  developed: 

a.  Reading  awareness : Paris'  Reading  Awareness  Questionnaire  was  utilized 
to  examine  students'  awareness  of  the  evaluation,  planning,  and  regulation 
skills  involved  in  reading. 

b.  Reading  strategies : A cloze  task  was  developed  to  evaluate  students'  reading 
comprehension  abilities  and  strategies.  It  is  a procedure  in  which  words 


are  systematically  omitted  and  students  are  required  to  fill  in  the  blanks 
with  the  appropriate  words. 

c.  Comprehension  monitoring : An  error  detection  task  was  designed  to  assess 
students’  comprehension  monitoring  skills.  This  task  requires  students  to 
detect  anomalous  information  in  the  passages;  their  awareness  of  faulty 
comprehension  is  taken  as  a measure  of  their  comprehension  monitoring. 

d.  Perceived  problem-solving  ability : Heppner  and  Petersen's  Problem- 
Solving  Inventory  (PSI)  was  adopted  to  examine  the  underlying 
dimensions  of  students’  perceptions  of  their  real-life,  personal  problem- 
solving process. 

e.  Problem  solving  strategies:  A problem-solving  task  was  developed  to 
assess  students'  problem-solving  strategies  in  mathematics.  This  task 
requires  students  to  think  aloud  (verbalize  their  thoughts)  as  they  try  to 
solve  the  given  math  problems. 

To  test  the  effects  of  the  experimental  programs,  most  of  the  above  measures 
were  implemented  in  both  the  pre-test  and  post-test  phases.  Measures  of  cognitive 
strategies,  however,  were  not  available  at  the  pre-test  point  and  were  thus 
administered  at  the  post-test  point  only. 

Assessment  of  participants'  perceptions 

In  addition  to  the  above  criterion  measures,  participants'  perceptions  with  respect  to 
their  involvement  in  the  Cognitive  Education  Project  were  also  assessed  by  survey 
questionnaires. 

Questionnaires  were  developed  to  ascertain  the  perceptions  of  the  teachers 
involved  in  each  condition  of  the  Cognitive  Education  Project  and  were  administered  on  an 
annual  basis.  The  purpose  of  these  questionnaires  was  to  determine  the  teachers' 
perceptions  of  the  project,  including  support  and  consultation  provided  by  the  team,  test 
administration  concerns,  usefulness  of  test  data  provided,  pupil  behavioral  change, 
inservice  effectiveness,  appropriateness  of  experimental  program  to  variation  in  grade, 
class  size,  and  time  allotted  for  strategy  instruction.  Also,  a follow-up  teacher 
questionnaire  was  administered  after  the  completion  of  the  study. 

A questionnaire  regarding  the  principal's  perception  of  both  the  implementation 
of  the  experimental  programs  and  participation  in  the  Cognitive  Education  Project  was 
distributed  to  all  principals  on  an  annual  basis.  A follow-up  survey  was  also 
administered  to  principals  of  experimental  condition  schools  after  the  completion  of  the 
project  to  determine  their  perceptions  regarding  their  involvement  in  the  study. 


With  respect  to  parents'  perceptions,  a questionnaire  was  sent  to  all  subjects’ 
parents.  In  this  questionnaire  parents  were  asked  if  they  recognized  any  positive 
changes  in  their  child  in  the  following  nine  behaviors:  attention  to  homework,  time  spent 
on  task,  ability  to  accept  criticism,  willingness  to  tackle  more  difficult  tasks, 
questioning,  alternative  points  of  view,  self-confidence,  originality  in  thinking,  and 
vocabulary. 

Summary  of  methodology 

The  study  involved  the  evaluation  of  two  learning/thinking  strategy  teaching  programs 
in  comparison  to  traditional  classroom  instruction  in  elementary  and  junior  high 
schools  in  north-central  Alberta.  Teachers  received  inservice  training  and  then  taught 
the  programs  for  a period  of  two  years.  Strategy  instruction  was  followed  by  one  year  of 
maintenance,  during  which  all  strategy  instruction  was  withheld.  Nine  hundred  pupils 
comprising  gifted,  average,  and  learning  disabled  students  were  initially  identified  to  be 
followed.  Change  with  respect  to  cognitive  ability,  reading  and  math  achievement,  affect, 
perceived  competence  and  cognitive  strategies  in  reading  and  math  was  evaluated  over  the 
course  of  the  three  years.  As  well,  the  perceptions  of  teachers,  administrators  and 
parents  with  respect  to  the  two  experimental  programs  were  assessed. 

Results 

The  results  of  the  study  are  discussed  in  terms  of  the  effects  of  the  two  cognitive 
education  programs  and  conventional  instruction  on  student  performance  in  four  major 
areas:  cognitive  ability,  academic  achievement,  affective  perceptions,  and  cognitive 
strategies.  The  major  analyses  were  two-way  ANOVAs  with  the  first  factor  being 
experimental  group  (I.E.,  S.P.E.L.T.,  Control)  and  the  second  a repeated  factor  being  the 
testing  periods. 

Grade  4 results 

In  the  following  sections,  the  results  are  reported  first  for  the  learning  disabled,  then 
for  the  average-achieving,  and  finally  for  the  gifted  students. 

Learning  disabled 

For  the  learning  disabled  students  at  the  grade  4 level,  a number  of  experimental 
program  effects  were  observed,  except  for  the  cognitive  ability  measure.  The  most 
pervasive  experimental  effects  were  observed  for  academic  achievement, 
particularly  in  the  area  of  reading  comprehension.  Figure  1 shows  that  both  I.E.  and 


S.P.E.L.T.  students  appear  to  display  greater  gains  in  reading  comprehension  grade 
scores  after  two  years  of  cognitive  instruction  when  compared  to  their  control 
counterparts.  However,  I.E.  students  appeared  to  regress  during  the  maintenance  period 
performing  at  essentially  the  same  level  as  Controls  by  the  end  of  the  three-year  period. 
S.P.E.L.T.  students  on  the  other  hand,  demonstrate  little  change  during  the  maintenance 
period  and  after  three  years  of  instruction  are  performing  at  about  one  grade  level 
higher  than  both  I.E.  and  Control  students. 

Figure  1 

Reading  Comprehension:  Grade  4 Learning  Disabled 


Time 

There  were  also  program  effects  observed  for  math  concepts  and  application.  The 
results  indicate  program  impact  as  both  the  I.E.  and  S.P.E.L.T.  groups  show  increased 
performance  over  the  Control  group  at  the  end  of  the  maintenance  year  (see  Figure  2). 


Figure  2 


Math  Concepts  and  Application:  Grade  4 learning  disabled 


The  standardized  reading  achievement  changes  were  coupled  with  changes  in  related 
strategy  areas  of  reading  most  notably  in  metacognitive  reading  awareness  (see 
Figure  3)  and  to  a lesser  extent  in  cloze  performance  and  comprehension  monitoring. 
These  related  effects  were  more  consistently  observed  for  S.P.E.L.T.  students.  This 
might  be  expected  since  many  of  the  reading  strategies  taught  in  the  S.P.E.L.T.  program 
were  more  directly  related  to  reading  tasks  than  are  those  in  the  I.E.  program. 

Figure  3 


Metacognitive  Reading  Awareness:  Grade  4 Learning  Disabled 


Also,  the  improvement  of  math  problem  solving  skills  of  the  I.E.  and  S.P.E.L.T. 
students  was  coupled  with  some  changes  in  the  frequency  of  math  strategies  used  in 
solving  individual  problems.  This  was  particularly  true  for  the  S.P.E.L.T.  students  who 
displayed  a greater  frequency  of  use  of  the  reread  strategy  as  well  as  the  strategy  stating 
a plan  during  problem  solution. 

In  terms  of  affective  perceptions,  there  were  no  significant  program  effects 
observed  with  respect  to  changes  in  students'  perceived  competence  or  self-concept. 
However,  experimental  effects  were  observed  with  respect  to  locus  of  control.  The 
S.P.E.L.T.  and  I.E.  students  displayed  an  increase  in  overall  internal  locus  of  control  after 
the  two  years  of  instruction. 

In  short,  the  general  effects  revealed  here  provide  a reasonably  optimistic 
picture  regarding  the  impact  of  cognitive  education  for  students  with  learning 
difficulties  in  regular  classrooms. 

Average-achieving 

For  grade  4 average-achieving  students,  the  results  indicated  that  the  groups  were 
performing  at  essentially  the  same  level  on  the  cognitive  ability  measure  at  both  pre  and 
post-test. 

With  respect  to  academic  achievement,  no  significant  program  effects  were 
observed.  Differences  between  groups  in  frequency  of  different  strategy  use  when 
solving  math  problems  were  observed.  For  example,  a significant  group  main  effect,  as 
well  as  a group-by-time  interaction,  was  obtained  for  the  strategy  of  "stating  plans". 
Figure  4 displays  the  results.  It  appears  that  the  interaction  is  due  to  the  behavior  of  the 
S.P.E.L.T.  students.  After  one  year  of  instruction  S.P.E.L.T.  students  displayed  a higher 
frequency  of  use  of  the  strategy  as  compared  to  both  I.E.  and  Control  students  and  this 
tended  to  endure  over  the  three  years.  There  was  also  a greater  overall  frequency  of  use 
of  a reread  strategy  observed  for  S.P.E.L.T.  students  as  compared  to  both  control  and  I.E. 
students. 
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Figure  4 

Math  Strategy  (Stating  Plans):  Grade  4 Average  Students 


Time 


With  regard  to  affective  perceptions,  little  effect  was  observed  with  respect  to 
self-concept  and  perceived  competence.  There  were,  however,  experimental  effects 
obtained  with  respect  to  locus  of  control.  After  the  two  years  of  instruction  both 
S.P.E.L.T.  and  I.E.  students  displayed  a greater  degree  of  internal  locus  of  control  than 
students  in  the  traditional  program. 

Gifted 

For  the  grade  4 gifted  students,  the  results  indicated  some  changes  in  student  performance 
in  terms  of  cognitive  strategies  in  the  area  of  reading  comprehension  and,  to  some 
extent,  in  math  problem  solving. 

Specifically,  the  reading  cloze  performance  of  both  I.E.  students  and  S.P.E.L.T. 
students  was  better  than  for  Controls  particularly  by  the  end  of  maintenance  year  (see 
Figure  5).  This  was  coupled  with  improved  comprehension  monitoring  for  both 
S.P.E.L.T.  and  I.E.  students  as  well  as  a trend  for  increased  metacognitive  reading 
awareness.  Ceiling  effects  were  observed  for  the  standardized  reading  measures,  thus  no 
significant  increases  could  be  expected  with  respect  to  the  standardized  measure  of 
reading. 


Figure  5 


Cloze  (Synonyms):  Grade  4 Gifted 


Figure  6 

Math  Strategy  (Determining  Alternative  Ways):  Grade  4 Gifted 


Time 

I.E.  and  S.P.E.L.T.  students  displayed  greater  frequency  in  the  use  of  the  strategy 
of  determining  alternative  ways  of  solving  a problem  as  well  as  the  strategy  of 
determining  the  reasonableness  of  an  answer  when  confronted  with  math  word  problems. 
For  the  strategy  of  determining  alternatives,  both  S.P.E.L.T.  and  I.E.  students 
demonstrated  greater  use  of  determining  alternative  ways  of  solving  problems, 


particularly  at  the  end  of  maintenance  (see  Figure  6).  This  is  clearly  the  case  for  I.E. 
students  who  displayed  an  increasing  frequency  of  use  of  this  strategy  over  time. 

Regarding  affective  perceptions,  there  were  no  significant  program  effects 
observed  with  respect  to  changes  in  students'  perceived  competence,  self-concept  or 
locus  of  control. 

Grade  7 results 

The  results  for  the  grade  7 students  are  presented  in  the  same  order  and  format  as  was 
the  case  for  the  grade  4 students. 

Learning  disabled 

For  the  learning  disabled  students  at  the  grade  7 level,  no  significant  program  effects 
were  observed  for  verbal  or  quantitative  scale  scores  of  the  cognitive  ability  measure. 
However,  a significant  group-by-time  interaction  was  obtained  with  respect  to  non- 
verbal ability.  The  interaction  appears  to  be  due  to  an  increase  in  non-verbal  ability 
for  the  S.P.E.L.T.  students  as  compared  to  I.E.  and  Control  students  who  performed 
essentially  at  the  same  level  or  decreased  slightly.  The  pre-test  to  post-test  means  for 
I.E.,  S.P.E.L.T.,  and  Control  were  98.8  to  101.3,  103.2  to  108.3,  and  98.5  to  99.8 
respectively. 

With  regard  to  academic  achievement,  trends  toward  program  effects  were 
observed  for  reading  comprehension  and  vocabulary,  the  findings  displayed  some  changes 
as  a result  of  program  implementation  and  this  was  particularly  true  for  S.P.E.L.T. 
students  (see  Figures  7 and  8). 


Figure  7 


Reading  Comprehension:  Grade  7 Learning  Disabled 


Figure  8 

Reading  Vocabulary:  Grade  7 Learning  Disabled 
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Changes  were  also  observed  after  two  years  of  instruction  for  standardized 
measures  of  both  math  computation  and  math  concepts  and  application.  I.E.  students 
performed  better  in  math  computation  than  either  S.P.E.L.T.  or  Control  students  at  the 
end  of  the  two  years  of  instruction,  but  this  difference  disappeared  at  the  end  of 


maintenance.  This  pattern  for  I.E.  students  was  also  observed  for  math  concepts  and 
application. 

For  the  measures  of  affective  perceptions,  the  experimental  programs  appear  to 
have  had  little  effect  on  the  perceived  competence,  perceived  problem  solving  ability, 
and  self-concept  of  adolescent  learning  disabled  students  over  a three-year  period  of 
time.  However,  the  degree  of  internal  locus  of  control  was  affected  for  students  in  the 
S.P.E.L.T.  program  more  so  than  I.E.  or  Control  students. 

In  terms  of  cognitive  strategies,  the  reading  and  math  related  strategy  measures 
displayed  some  changes  as  a result  of  program  implementation.  This  was  particularly 
true  for  S.P.E.L.T.  students.  Although  the  results  were  not  consistently  significant,  the 
majority  of  trends  indicated  changes  in  favor  of  the  experimental  students. 

Average-achieving 

There  were  few  obvious  experimental  effects  observed  for  grade  7 average  achieving 
students.  There  was  some  indication  of  increased  reading  performance  in  using  contextual 
strategies  in  reading  after  the  three  years  (as  measured  by  the  cloze  task)  for  S.P.E.L.T. 
students  as  compared  to  I.E.  and  Control  students.  Few  other  significant  effects  were 
observed. 

Gifted 

The  results  for  the  grade  7 gifted  students  displayed  some  interesting  patterns.  There 
were  some  changes  in  achievement  in  math  computation  and  math  concepts  and  application. 
The  standardized  math  computation  performance  of  the  I.E.  students,  and  to  a lesser  extent 
the  S.P.E.L.T.  students,  appeared  to  be  affected  after  two  years  of  instruction.  The  gains 
over  the  two  years  for  I.E.,  S.P.E.L.T.,  and  Control  students  were  2.9,  3.0  and  2.6, 
respectively.  These  differences  tended  to  disappear  at  maintenance  with  Controls  doing 
better.  However,  ceiling  effects  were  beginning  to  play  a part  at  both  the  end  of  two  years 
and  at  maintenance.  The  math  concepts  and  application  performance  appeared  also  to  be 
affected  to  some  extent  by  both  I.E.  and  S.P.E.L.T.  teaching.  The  gains,  after  two  years  of 
instruction  for  I.E.,  S.P.E.L.T.,  and  Control,  were  grade  equivalents  of  2.0,  2.0  and  1.1, 
respectively. 

With  respect  to  reading  performance,  there  were  no  significant  effects  observed 
for  standardized  measures  of  reading.  This  is  to  be  expected  since  the  level  of  performance 
by  all  of  the  gifted  students  was  high  at  the  outset. 

There  was,  however,  evidence  for  positive  change  in  metacognitive  reading 
strategy  awareness  for  both  I.E.  and  S.P.E.L.T.  students.  The  means  for  I.E.,  S.P.E.L.T.  and 
Control  students  were  34.6,  34.9  and  31.5,  respectively.  Overall  the  two  cognitive 


20 


education  programs  appeared  to  significantly  affect  gifted  grade  7 students' 
metacognitive  awareness  in  reading.  After  approximately  six  months  of  instruction, 
the  metacognitive  reading  awareness  of  these  students  appeared  to  be  enhanced  by  both 
experimental  programs  and  this  difference  was  maintained  over  time  (see  Figure  9). 

Figure  9 


Metacognitive  Reading  Awareness:  Grade  7 Gifted 
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The  math  strategies  students  chose  to  use  in  solving  particular  math  problems 
also  demonstrated  some  experimental  effects  particularly  for  I.E.  students  and  to  a lesser 
extent  S.P.E.L.T.  students.  For  example,  a significant  group-by-time  interaction  was 
obtained  for  the  frequency  of  usage  of  the  rereading  strategy.  At  the  post  year  2 
point,  while  Control  and  I.E.  increased,  the  S.P.E.L.T.  score  remained  relatively  stable. 

At  maintenance,  the  interaction  was  due  to  increasing  I.E.  scores  compared  to  decreasing 
S.P.E.L.T.  and  Control  scores.  The  I.E.  students  displayed  greater  usage  of  the  rereading 
strategy  as  compared  to  both  Control  and  S.P.E.L.T.  students  after  maintenance  (see 
Figure  10). 
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Figure  10 

Math  Strategy  (Rereading):  Grade  7 Gifted 


Finally,  with  respect  to  affective  perceptions,  no  significant  program  effects 
were  observed  for  any  of  the  three  measures  of  affective  perceptions. 


Participants'  Perceptions 

In  order  to  monitor  and  evaluate  the  evolving  perceptions  of  each  of  the  participating 
groups  (teachers,  principals  and  parents),  with  respect  to  their  involvement  in  the 
Cognitive  Education  Project,  survey  questionnaires  were  developed  and  administered  to 
each  group  on  an  annual  basis  for  three  years.  Presented  here  are  summaries  of  the 
information  gathered  from  these  surveys. 

Teachers'  perceptions 

The  perceptions  of  the  participating  teachers  were  extremely  positive  for  both  I.E.  and 
S.P.E.L.T.  Immediately  at  the  end  of  implementation  and  project  involvement,  all 
teachers  in  S.P.E.L.T.  and  the  majority  of  I.E.  teachers  indicated  that  they  would  like  to 
continue  using  the  procedures.  When  asked  one  and  a half  to  two  years  later,  a 
significant  number  of  teachers  (85%)  indicated  they  were  indeed  teaching  certain 
aspects  of  the  program  to  their  classes.  All  the  teachers  in  I.E.  and  S.P.E.L.T.  indicated 
that  they  would  recommend  their  respective  programs  to  their  colleagues  and 
furthermore,  that  involvement  in  the  training  had  enhanced  their  professional 
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development.  The  inservice  training  and  follow-up  support  were  also  considered  to  be 
appropriate.  In  addition,  some  teachers  indicated  that  more  inservice  days  spread  over 
the  year  after  the  initial  three-day  inservice  would  have  been  more  beneficial.  Teachers 
generally  felt  that  both  experimental  programs  were  effective  for  all  student  ability 
levels,  although  some  I.E.  junior  high  teachers  indicated  that  the  program  was  most 
appropriate  for  low  ability  students. 

Principals'  perceptions 

Very  few  differences  were  noted  between  the  questionnaire  responses  given  by  the 
S.P.E.L.T.  and  the  I.E.  principals.  Generally,  principals  involved  in  the  experimental 
conditions  viewed  their  teachers  as  being  relatively  enthusiastic  about  the  project  both 
at  the  conclusion  of  the  inservice  and  over  the  course  of  the  year. 

Problems  were  noted  for  scheduling  and  implementation  in  the  I.E.  condition. 
While  I.E.  principals  encountered  difficulties  in  the  scheduling  and  implementing  of  the 
required  three  forty-minute  classes  outside  of  the  regular  curriculum,  this  was  not  the 
case  for  S.P.E.L.T.  principals.  I.E.  principals  reported  both  personal  and  teacher 
frustration  arising  from  these  timetabling  difficulties.  Such  problems  apparently 
affected  grades  7 and  8 more  frequently  than  the  grades  4 and  5 classes. 

Parents'  perceptions 

The  perceptions  of  parents  with  respect  to  changes  in  their  children  suggested  that  there 
were  generalizing  effects  being  observed  at  home.  A number  of  behavioral  changes  were 
related  and  were  consistently  being  reported  more  often  by  I.E.  and  S.P.E.L.T.  parents 
than  Controls.  The  most  frequently  reported  behavioral  changes  over  grades  were  in 
attention  to  homework,  in  recognizing  alternative  points  of  view,  in 
willingness  to  tackle  more  difficult  tasks,  in  self-confidence,  and  in 
questioning.  The  parent  data  strongly  support  the  idea  that  program  effects  are  being 
generalized  for  both  S.P.E.L.T.  and  I.E.  students,  particularly  after  one  year  of 
instruction  and  at  the  end  of  maintenance. 
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Summary  of  Results 


Student  change 

The  results  appear  very  promising,  particularly  for  learning  disabled  students  and  to  a 
somewhat  lesser  extent  gifted  students  (see  Table  2).  The  most  pronounced  effects  were 
observed  for  the  grade  4 learning  disabled  students,  most  notably  in  reading 
comprehension  and  related  strategies.  There  was  also  evidence  that  the  average  and 
gifted  students  benefited,  but  to  a lesser  degree.  Generally,  S.P.E.L.T  instruction  tended 
to  produce  more  changes  as  compared  to  I.E.  and  Control.  This  finding  was  not  unexpected 
since  the  S.P.E.L.T.  instruction,  in  large  part,  involves  teaching  cognitive  strategies 
directly  within  curricular  content  areas. 

The  lack  of  consistent  maintenance  of  behavioral  change,  which  was  sometimes 
observed  for  I.E.  students,  may  be  due  to  insufficient  time  allotted  for  I.E.  instruction. 
Maintenance  of  the  program  might  well  be  achieved  if  I.E.  instruction  could  have  been 
continued  for  a longer  period  of  time. 

This  study  is  a highly  conservative  one,  as  in  most  cases  only  one  or  two  teachers 
in  a school,  at  one  grade  and  subject  level,  taught  either  the  I.E.  or  S.P.E.L.T.  program. 

If  all  teachers  at  each  grade  level  were  engaged  in  the  teaching  of  cognitive  education 
procedures,  quicker  and  more  comprehensive  effects  would  likely  have  emerged.  There 
is  some  evidence  for  this  expectation  in  a study  conducted  in  Barrhead  and  Swan  Hills 
schools  in  1985  (Mulcahy,  Peat,  & Darko-Yeboah,  1986).  All  teachers  from  grades 
4 to  12  were  trained  in  cognitive  education  procedures,  with  pre-  and  post-tests 
administered  to  students  during  the  initial  year.  Significant  pre-post  gains  were  noted 
on  measures  of  self-concept,  perceived  problem  solving,  metacognitive  reading 
awareness,  and  strategy  use. 

The  results  obtained  in  the  present  study  are  also  consistent  with  findings  from 
recent  research  on  the  teaching  of  metacognitive  strategies  (Paris  & Oka,  1986). 

Haller,  Child  and  Walberg  (1988)  conducted  a meta-analysis  of  twenty  studies  of 
metacognitive  strategy  teaching  with  respect  to  reading  comprehension  which  involved  a 
total  of  1,553  students.  They  obtained  an  average  effect  size  of  .71  which  they  report  as 
one  of  the  largest  uncovered  in  educational  research  to  date.  This  supports  the  claim  that 
metacognitive  strategy  teaching  is  effective  in  making  children  more  aware  of  reading 
strategy  variables. 


Table  2 

Summary  Chart  of  the  Three-Year  Results 


Variable 

Grade 

Program  Effect 
Learning 

Disabled  Average  Gifted 

Cognitive  Ability 

4 

No 

No 

No 

7 

No 

No 

No 

Academic  Achievement 

Math  Computation 

4 

No 

No 

No 

7 

Yes(1) 

No 

Yes(1 ,2) 

Math  Concepts  and  Application 

4 

Yes(1 ,2) 

No 

No 

7 

Yes(1 ) 

No 

Yes(i  ,2) 

Reading  Vocabulary 

4 

No 

No 

No 

7 

No 

No 

No 

Reading  Comprehension 

4 

Yes(2) 

No 

No 

7 

Trends(2) 

No 

No 

Affective  Perceptions 

Perceived  Competence 

4 

No 

No 

No 

7 

No 

No 

No 

Self  Concept 

4 

No 

No 

No 

7 

No 

No 

No 

Locus  of  Control 

4 

Yes(1 ,2) 

Yes(2) 

No 

7 

No 

No 

No 

Cogniti  ve  S tra  tegies 

Reading  Strategies  Awareness 

4 

Yes(1 ,2) 

No 

Trend 

7 

No 

No 

Yes(1 ,2) 

Reading  Cloze  Performance 

4 

Yes(2) 

No 

Yes(1 ,2) 

7 

Yes(2) 

Yes(2) 

Yes(1 ,2) 

Comprehension  Monitoring 

4 

Yes(2) 

No 

Yes(1 ,2) 

7 

Yes(1 ,2) 

No 

Yes(1 ,2) 

Math  Problem  Solving  Strategies 

4 

Yes(2) 

No 

Yes(1 ,2) 

7 

Yes(1 ,2) 

No 

Yes(1 ) 

Perceived  Problem  Solving  Ability 

4 

No 

No 

No 

7 

No 

No 

No 

1=  I.E.,  2=S.P.E.L.T.,  3=C0NTR0L 
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Participants'  perceptions 

The  perceptions  of  parents,  teachers  and  administrators  regarding  the  two  cognitive 
education  programs  are  extremely  encouraging.  The  vast  majority  of  experimental 
teachers  reported  that  they  would  continue  to  use  the  instructional  procedures  from  the 
two  programs.  Indeed  when  a significant  number  was  surveyed  again  in  November 
1988,  two  years  after  involvement  in  the  project,  over  85%  of  the  teachers  reported 
they  were  continuing  to  use  aspects  of  the  programs  in  their  teaching.  This  was  coupled 
with  the  fact  that  all  teachers  indicated  they  would  still  recommend  the  programs  to 
their  colleagues. 

Parents'  responses,  although  limited  to  the  second  and  third  years,  also  indicated 
positive  changes  in  their  youngsters,  in  a number  of  important  behavioral  areas  such  as 
self-confidence,  task  persistence,  accepting  alternative  points  of  view, 
originality  of  thinking,  and  questioning. 

Administrators'  perceptions  of  the  two  experimental  programs  were  also 
generally  very  positive.  Some  concern  was  evident  with  respect  to  the  high  cost  of  I.E. 
materials  as  well  as  the  practice  of  teaching  I.E.  in  isolation  from  the  rest  of  the 
curriculum.  However,  on  follow-up  questionnaires,  one  and  one-half  to  two  years  after 
their  involvement  in  the  initial  implementation,  over  half  of  the  24  principals  said  they 
would  consider  adopting  the  experimental  programs.  Many  of  these  principals  stated 
they  would  also  recommend  the  cognitive  education  programs  to  other  schools. 

Inservice  training 

With  respect  to  the  inservice  training  provided  for  the  two  experimental  programs,  the 
vast  majority  of  the  teachers  indicated  that  inservice  training  was  sufficient  to  allow 
them  to  implement  the  programs  adequately.  There  was  some  indication  from  a number 
of  teachers  that  more  "in-school  support"  would  have  enhanced  implementation  of  the 
programs.  A feeling  of  isolation  (being  the  only  teacher  in  the  school  involved),  was 
voiced  on  occasion  by  some  teachers.  The  administrators  and  other  staff  were  not 
familiar  with  the  programs.  Nevertheless,  teachers  were  able  to  learn  and  implement 
the  programs  with  some  degree  of  facility.  The  need  to  have  all  staff  become  familiar  and 
involved  from  the  beginning  of  the  program  was  evident. 

The  teachers  in  both  experimental  programs  indicated  the  desirability  for 
additional  follow-up  inservice  sessions  over  the  years.  This  could  be  handled  to  a 
considerable  extent  through  peer  coaching  and  staff  meetings  devoted  to  discussion  of 
teaching  procedures,  and  generalization  over  content  areas  and  grades. 
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Implications 

Cognitive  education  as  a part  of  school  curriculum 

The  results  observed  with  respect  to  pupil  change,  coupled  with  the  perceptions  of 
parents,  teachers  and  administrators,  suggest  that  the  teaching  of  learning/thinking 
strategies  should  be  made  an  integral  part  of  the  school  curriculum. 

The  question  of  whether  the  cognitive  education  programs  are  more  appropriate 
at  different  grade  levels  or  for  different  diagnostic  groups  could  not  be  answered 
definitively.  However,  instruction  at  the  lower  grades  was  associated  with  better  gains 
than  instruction  at  the  higher  grades.  The  results  clearly  indicate  that  both  programs 
have  a greater  effect  on  students  at  the  grade  4 level.  The  teachers  involved  in  the 
experimental  conditions  also  indicated  general  appropriateness  of  both  programs  for 
grade  4 students.  Both  programs  appear  to  be  most  effective  for  grades  4 and  7 learning 
disabled  students  and  to  a lesser  extent  for  gifted  students.  It  is  somewhat  puzzling  that 
there  appeared  to  be  less  impact  on  average  students  as  compared  to  learning  disabled  or 
gifted  students.  This  may  be  due  to  the  fact  that  learning  disabled  students  generally  lack 
a systematic  strategic  approach  to  tasks  and  thus  benefit  more  quickly  when  provided 
with  a systematic  approach.  Average  students  may  already  have  a somewhat  effective 
approach  in  place,  and  thus  fail  to  benefit  significantly  from  the  programs  at  the  outset. 
Gifted  students  have  the  intellectual  ability  to  perceive  the  usefulness  of  the  strategies 
and  then  to  use  and  extend  them  immediately.  Many  of  the  teachers  commented  that  they 
found  the  higher  ability  students  to  "take  off'  with  the  strategies  in  extending  and 
applying  them. 

Cognitive  education  for  students 

The  results  have  clear  implications  for  the  mainstreaming  of  students  with  learning 
difficulties,  as  well  as  gifted  students.  The  impact  of  the  teaching  of  cognitive  strategies 
on  the  learning  disabled  students,  particularly  at  grade  4,  suggests  that  if  the  teaching 
approaches  are  used  systematically  throughout  the  elementary  school,  it  may  prevent 
some  students  from  developing  severe  learning  problems,  and  keep  them  in  the 
mainstream. 

The  recent  research  on  the  teaching  of  learning/thinking  strategies  to  learning 
disabled  students  also  demonstrates  significant  effect  with  respect  to  achievement  (see 
for  instance  Palincsar  & Brown,  1984;  Paris  & Oka,  1989;  Goldman,  1989).  The 
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effects  observed  in  this  study  for  gifted  students  also  indicate  that  they  may  benefit  from 
such  teaching  in  the  regular  classroom. 

These  approaches  should  also  be  effective  for  mildly  mentally  retarded  as  well  as 
native  youngsters  in  regular  classrooms.  Indeed  some  researchers  have  suggested  this 
(Mulcahy  & Marfo,  1987;  Brown,  1984).  There  is  a need  for  further  research  on 
teaching  learning/thinking  strategies  for  these  populations.  Further  investigation  into 
the  effects  of  extending  cognitive  instruction  to  primary,  senior  high  and  post-secondary 
levels  is  also  required.  Current  research  at  the  preschool  level  with  high  risk  children, 
appears  to  hold  significant  promise  (Haywood,  Brooks,  & Burns,  1986;  Price,  1991). 

Assessment  of  cognitive  strategies 

The  two  cognitive  education  programs  affected  the  cognitive  strategies  utilized  by 
students  after  instruction  as  well  as  increased  their  degree  of  metacognitive  reading 
awareness.  The  comprehension  monitoring  skills  of  students  after  two  years  of 
instruction  in  the  two  cognitive  education  programs  were  observed  to  be  generally  better 
than  their  control  counterparts,  and  so  was  their  performance  on  a cloze  reading  task. 
Both  these  tasks  require  students  to  use  context  to  fill  in  missing  words  and  to  determine 
the  comprehensibility  of  the  text.  As  Paris  and  Oka  (1986)  have  suggested,  skimming, 
rereading,  paraphrasing,  inferring  and  checking  are  strategies  that  one  would  use  on 
these  tasks.  No  direct  measures  of  these  particular  strategies  were  taken  during  the 
Cognitive  Education  Project.  However,  the  changes  in  math  strategies  noted  in  I.E.  and 
S.P.E.L.T.  students  were  often  those  of  rereading,  stating  plans  and  determining 
alternative  solutions. 

The  assessment  of  cognitive  strategies  was  addressed  to  some  extent  by  the 
project.  However,  it  may  be  that  more  pervasive  strategic  change  could  have  been 
identified  if  more  appropriate  criterion  measures  of  cognitive  strategies  were  available. 
Further  research  must  address  the  development  of  these  instruments  for  researcher  as 
well  as  practitioner  use.  The  math  problem  solving  strategy  approach,  the  Metacognitive 
Reading  Strategy  Awareness  Inventory,  and  to  some  extent,  the  Perceived  Problem 
Solving  Inventory,  used  in  this  study  appear  to  hold  some  promise  both  from  face  validity 
and/or  from  results  reported  here  and  elsewhere. 
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Reflections  of  the  Research  Team 

The  study  reported  here  is  a very  extensive  one  involving  an  extremely  large  number  of 
participants.  When  one  is  involved  with  such  a large  number  of  individuals  over  an 
extended  period  of  time,  there  is  a great  deal  of  what  might  be  termed  qualitative  data 
collected  which  can  shed  further  light  on  the  study  about  cautions  which  future  studies  of 
this  nature  can  benefit  from.  The  following  provides  some  of  this  qualitative  data  from  the 
perspective  of  the  research  team.  There  are  essentially  six  major  topics  of  importance 
related  to  the  study  which  we  feel  merit  attention:  data  interpretation,  inservice  training, 
implementation,  program  management,  evaluation,  and  programs. 

Some  considerations  regarding  interpretation 

As  with  any  study  there  are  significant  limitations  which  must  be  considered  in 
interpreting  the  results  reported.  Among  the  more  critical  with  regard  to  this  study 
are:  with  the  large  number  of  statistical  analyses  significant  results  by  chance  are 
increased;  specific  tests  of  program  mastery  were  not  conducted  independently  by  the 
research  team  so  it  is  possible  that  all  students  may  not  have  mastered  the  content  of  the 
experimental  programs;  the  majority  of  situations  involved  only  one  experimental 
teacher  in  a school  thus  restricting  the  instruction  to  one  classroom  of  students  as  well 
as  the  consolidation  and  generalization  of  the  learning  and  thinking  strategies;  the  small 
number  of  parent  questionnaire  returns  in  years  two  and  three  restricts  the  degree  of 
confidence  that  can  be  placed  on  these  results,  however,  the  results  at  the  end  of  one  year 
of  instruction  can  be  viewed  with  confidence  in  light  of  relatively  high  rates  of  return; 
not  all  students  had  all  measures  available  for  analysis,  however,  missing  data  due  to 
attrition,  test  administration,  etc.,  is  a significant  problem  and  is  particularly  a 
difficulty  in  longitudinal  studies;  the  use  of  grade  equivalent  scores  as  the  dependent 
measures  in  some  of  the  analyses  could  be  criticized. 

Inservice  training 

Due  to  the  large  number  of  requests  for  inservice  training  in  S.P.E.L.T.,  there  has  been 
much  opportunity  to  modify  and  refine  the  S.P.E.L.T.  inservice  model.  As  well,  through 
classroom  observations  throughout  the  four  years  of  the  Cognitive  Education  Project, 
insights  were  gained  as  to  which  aspects  of  inservice  training  were  applied  most 
frequently  at  the  classroom  level.  The  synthesis  of  this  information  is  presented  below. 

S.P.E.L.T.'s  three-phase  approach  to  the  teaching  of  learning/thinking  strategies 
appears  to  be  more  important  than  the  choice  or  sequence  of  strategies  used.  This 
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observation  is  strengthened  by  recent  research  articles  which  emphasize  'informed 
strategy  training'  (for  example,  see  Nickerson,  1988).  As  well,  there  was  a tendency 
for  some  teachers  in  the  Cognitive  Education  Project  S.P.E.L.T.  condition  to  remain 
within  the  direct  teaching  of  Phase  I for  extended  periods  of  time,  rather  than  moving  to 
Phases  II  and  III.  In  light  of  these  factors,  and  based  upon  the  responses  of  hundreds  of 
teachers  to  the  inservice  training  process,  the  inservice  training  model  has  been 
modified  as  follows: 

a)  Rather  than  spending  each  of  the  three  days  of  training  on  each  phase  of  the 
S.P.E.L.T.  instructional  model  sequentially  (i.e.,  Day  1=Phase  I;  Day  2=Phase  II;  Day 
3=Phase  III),  the  strategies  are  now  presented  in  content-based  clusters,  with  each 
cluster  moving  through  the  three  phases.  This  results  in  teachers  undergoing  an 
experience  with  the  S.P.E.L.T.  instructional  continuum  a minimum  of  four  times 
throughout  the  three-day  training  process.  Although  the  number  of  strategies  presented 
is  reduced  using  this  model,  the  importance  of  the  instructional  continuum  is 
emphasized. 

b)  The  emphasis  on  the  instructional  continuum,  with  multiple  strategy  clusters 
being  presented,  rather  than  the  individual  strategies,  seems  to  eliminate  the  tendency 
of  some  teachers  to  emphasize  only  one  area  of  strategy  application  such  as  reading 
comprehension  or  memory.  In  other  words,  the  breadth  of  strategies  used  in  classrooms 
appears  to  be  greater  than  when  the  original  inservice  training  model  was  used. 

The  five-day  inservices  for  I.E.  should  be  split  into  two-  and  three-day  blocks  in 
order  for  teachers  to  try  the  principles  of  I.E.  before  completing  the  entire  training.  In 
addition,  two  half-day  workshops  held  at  regular  intervals  would  give  teachers  a chance 
to  share  problems  and  ideas  and  to  renew  their  enthusiasm  with  respect  to  the  program. 

In  order  to  ensure  that  teachers  are  able  to  acquire  the  necessary  teaching  skills 
and  strategies  regardless  of  the  program,  cognitive  coaching  should  be  made  an  integral 
part  of  the  initial  training.  This  coaching  for  application  would  involve  hands-on,  in- 
classroom assistance  and  cognitive  modelling  to  transfer  skills  and  strategies  to  the 
classroom.  This  could  be  achieved  through  the  extended  training  of  classroom 
consultants  who  could  then  work  with  teachers  in  the  classroom  on  a continual  basis 
after  the  initial  inservice  training.  Peer  coaching  should  also  be  made  an  integral  part 
of  the  inservice  training  and  extended  into  the  classroom  situation. 
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Implementation 

The  question  of  whether  the  programs  were  implemented  as  intended,  was  determined 
through  on-site  visits  and  phone  calls.  In  the  majority  of  cases  the  programs  appeared 
to  have  been  implemented  adequately;  however,  it  is  clear  that  some  teachers  were 
implementing  to  a greater  degree  of  intensity  than  others.  Teacher  enthusiasm  with 
respect  to  the  two  particular  approaches  was  extremely  high,  so  it  was  difficult  to 
control  for  the  Hawthorne  effect.  However  as  has  been  pointed  out  by  other  researchers, 
it  is  unrealistic  to  expect  these  teachers  to  hide  their  enthusiasm.  Moreover,  teacher 
enthusiasm  is  useful  evaluation  data  (Nickerson,  1986)  indicating  face  validity  of  the 
programs. 

It  is  also  possible  in  the  present  study  that  the  control  group’s  performance  was 
enhanced  as  a result  of  participation  in  the  project.  There  were  some  indications  from 
control  teachers  to  suggest  this.  One  particular  comment  voiced  to  the  principal 
researcher  was,  "I  bet  my  students  will  do  as  good  or  better  than  those  students  being 
taught  strategies",  suggesting  that  some  control  teachers  may  have  viewed  themselves  in 
competition  with  the  experimental  groups.  Even  with  the  careful  monitoring  it  is  still 
possible  that  programs  were  not  as  systematically  implemented  as  intended.  In  order  to 
ensure  maximum  implementation  it  would  be  helpful  to  have  peer  teacher  coaching  made 
an  integral  part  of  the  procedure.  The  first  year  could  be  a program  implementation 
year  and  the  following  years,  evaluation  years. 

When  future  studies  of  this  nature  are  considered,  it  would  be  more  appropriate 
to  have  the  initial  year  as  an  implementation  year  where  teachers  are  trained  and 
monitored  through  the  year  and  the  coaching  is  developed  as  an  integral  part  of  the 
program.  The  succeeding  years  would  then  become  evaluation  years  which  would  result 
in  less  confusion  and  more  direct  evaluation  of  program  effects. 

When  inservice  training  pre-testing  and  implementation  are  all  occurring  at  the 
same  time,  teachers,  students  and  administrators  as  well  as  the  research  team  are 
dividing  attention  and  efforts  towards  many  variables  and  as  a result  one  or  more  of  these 
areas  suffers.  If  participants  are  able  to  concentrate  on  one  aspect  at  a time,  it  is  likely 
that  more  appropriate  implementation  might  occur.  Allowing  the  initial  year  to  enable 
teachers  to  concentrate  on  implementing  the  procedure  would  result  in  less  confusion. 
This  would  also  enable  the  research  team  to  engage  in  more  in-class  coaching  and 
monitoring  to  assist  in  more  appropriate  implementation.  This  raises  another  problem 
that  we  ran  into  in  the  implementation  which  was  a major  concern  for  the  teachers  as 
well  as  for  the  research  team.  The  feedback  from  a number  of  teachers  in  the  field 
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indicated  that  they  felt  they  were  being  torn  in  a number  of  directions  simultaneously  and 
thus  were  unable  to  put  forth  a concentrated  effort  to  the  implementation.  One  issue  was 
that  many  districts  had  a particular  focus  one  year  with  respect  to  the  teaching  and 
enhancing  effectiveness  and  the  next,  a new  thrust  or  focus  would  emerge.  An  example 
might  be  the  emphasis  on  teacher  effectiveness  training  and  then  next  year  focus  on 
increasing  students’  self-esteem.  What  this  tended  to  do  was  restrict  teachers  in  the 
ability  to  implement  the  strategy  teaching  consistently  over  the  years  with  a focused 
thrust.  The  comment  was  often  heard,  "we  are  being  asked  to  do  too  many  things  at  the 
same  time  with  not  a long  enough  period  with  any  one  focus  to  enable  us  to  do  the  task 
adequately".  The  research  is  clear  that  the  teaching  of  learning/thinking  strategies  is  not 
a brief  task  but  one  that  requires  a long-term  commitment  over  a number  of  years. 

The  experience  gained  through  implementing  this  project  pointed  out  a number  of 
pitfalls  that  need  to  be  avoided  when  schools  or  school  systems  contemplate  putting  in 
place  a program  of  learning/thinking  strategy  instruction.  Some  of  these  have  also  been 
documented  by  Chance  (1986). 

There  was  often  a tendency  to  move  too  fast  or  try  to  do  too  much  in  a short  period 
of  time.  It  was  evident  that  in  many  instances  too  many  cognitive  strategies  were 
attempted  to  be  taught  in  a very  short  time.  It  is  important  to  focus  on  a few  strategies 
over  the  course  of  a year  or  term  and  teach  them  well. 

It  appeared  in  many  cases  that  expectations  for  change  were  too  high.  The 
expectation  of  significant  changes  in  a short  period  of  time  could  have  led  to  some 
disenchantment  and  the  abandonment  of  the  teaching  or  at  the  most  a halfhearted  effort. 
Cognitive  strategy  instruction  is  a long-term  commitment  and  must  be  viewed  as  such. 

Giving  in  to  early  failures  can  be  a common  problem.  Instead  of  viewing  these  as 
failures,  it  is  important  to  treat  them  as  opportunities  to  learn  more  about  cognitive 
strategy  teaching.  In  these  early  stages  of  the  development  of  cognitive  instruction, 
mistakes  and  failures  are  to  be  expected  and  can  be  the  vehicles  for  adding  to  our 
knowledge  regarding  how  best  to  deliver  cognitive  education  instruction.  Just  as  we 
encourage  students  to  use  errors  in  their  problem  solving  to  assist  them  in  their 
learning,  so  too  should  teachers  of  cognitive  education. 

Program  management 

Record  keeping,  collecting  and  scoring  tests,  and  computerizing  results  were  onerous 
tasks.  The  recording  of  what  decisions  had  been  made  and  why  with  regard  to  testing, 
inservice,  questionnaire  distributions,  scoring,  student  attrition,  assessment 
difficulties,  and  missing  data  to  list  but  a few  is  a full-time  job.  It  is  necessary  to  keep 
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very  careful  written  records  of  dates,  times,  decisions  and  individuals  involved  in  order 
to  be  able  to  interpret  study  data  in  a realistic  way.  This  is  something  that  we  in  the 
study  team  did  as  consistently  as  possible.  However,  there  were  still  times  it  was  not 
totally  clear  what  had  happened  and  why. 

Communication  in  a project  of  this  nature  is  one  of  the  key  elements  which  can 
make  or  break  the  study.  It  is  essential  that  all  major  participants  are  kept  informed 
from  beginning  to  end  with  respect  to  all  aspects  of  the  study.  With  a study  of  this 
magnitude,  this  is  an  extremely  difficult  task.  At  the  very  beginning  in  order  to  make 
explicit  as  possible  each  participant's  role,  a summary  of  the  project  was  written  up 
including  questions  to  be  addressed  along  with  methodology.  The  particular  roles  and 
responsibilities  of  the  school  jurisdiction,  teachers,  and  research  team  were  clearly 
defined  in  this  summary  which  was  then  provided  to  participants  prior  to  involvement. 

A signed  written  contract  from  the  participating  school  jurisdictions  was  then  requested 
to  ensure  to  the  degree  possible,  the  continued  commitment  for  the  three-year  duration 
of  the  study.  As  well,  some  meetings  were  held  with  school  boards,  parents,  and/or 
administrators  to  communicate  the  study’s  intent.  In  future  studies,  meetings  of  this 
nature  should  be  an  integral  part  of  initial  program  management  to  ensure 
communication  is  appropriate  and  to  address  any  questions  or  concerns. 

Over  the  course  of  the  study  it  is  essential  to  ensure  ongoing  communication 
between  the  research  team  and  field.  We  attempted  to  do  this  by  requesting  one  contact 
person  (usually  an  assistant  or  associate  superintendent)  to  be  assigned  to  communicate 
with  the  research  team  and  teachers.  In  addition  phone  calls  and  newsletters  were  used  to 
maintain  contact  with  schools  up  to  80  to  100  miles  away.  In  future,  it  may  be  useful  to 
use  a computer  networking  system  with  the  schools  and  set  up  electronic  bulletin  boards 
where  sharing  of  information  could  occur  between  the  team  and  the  field  as  well  as 
between  teachers  in  the  field  involved  in  the  same  program.  This  would  allow  a sharing 
of  ideas  with  respect  to  program  implementation  as  well  as  helping  to  maintain  a focus 
on  the  implementation. 

Following  a small  number  of  children  out  of  a total  classroom  is  extremely 
difficult,  particularly  over  more  than  one  year.  Testing  becomes  disruptive  to  the 
entire  class  and  teachers  are  unable  to  conduct  group  testing  easily  as  they  have  to  divide 
their  time  between  those  students  being  followed  and  the  rest  of  the  class. 

When  following  these  students  into  the  next  grade,  significant  difficulties  were 
encountered  with  respect  to  both  students  and  teacher.  Typically  students  are  reassigned 
to  classrooms  at  the  end  of  each  year  which  means  that  a new  group  of  students  is  formed 
each  year.  Thus  some  of  the  students  had  been  taught  the  program  for  one  year  whereas 
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others  had  no  knowledge  of  the  program.  In  the  project  we  attempted  to  have  students 
move  as  a complete  group  to  the  next  class,  however,  this  was  only  achieved  in  a very 
small  number  of  cases  at  grade  4.  If  this  was  not  possible  then  those  students  who  were 
being  followed  were  kept  together  as  a group  in  their  new  class.  The  teacher  then 
reviewed  the  previous  year's  work  emphasizing  those  aspects  crucial  to  the  program 
movement  to  the  next  level.  Identifying  teachers  at  the  beginning  who  would  become 
involved  in  the  second  year  and  training  them  in  the  initial  year  would  be  extremely 
helpful  in  having  teachers  ready  to  teach  the  next  level  of  the  programs.  If  all  teachers 
were  involved  this  would  not  be  as  much  of  a problem. 

All  of  the  testing  would  best  be  done  by  the  research  team  to  ensure  consistency 
as  well  as  greater  validity  and  reliability  of  data.  Teachers  might  best  administer  only 
those  tests  directly  usable  and  interpretable  by  them. 

Evaluation 

In  studies  of  this  nature,  the  question  of  the  most  appropriate  criterion  measures  is  a 
major  concern.  We  attempted  to  evaluate  the  program  from  a number  of  perspectives 
including  the  cognitive  strategies  employed  by  the  students.  There  is  a need  in  the 
continuing  work  of  this  nature  to  further  develop  instruments  and  techniques  more 
sensitive  to  the  development  of  learning/thinking  strategies  and  their  transfer. 

In  evaluating  the  implementation  it  is  obvious  to  us  that  we  have  not  been  able  to 
detect  changes  that  may  have  occurred.  Teachers  would  often  comment  that  the 
implementation  of  the  programs  had  positively  changed  their  teaching  style  and  the  way 
they  now  viewed  and  interacted  with  their  students.  Many  teachers  teaching 
Instrumental  Enrichment  for  example,  indicated  when  visited  in  their  classrooms  that 
students  who  previously  would  not  volunteer  an  answer  in  class  or  debate  a point  would 
now  do  so,  thus  increasing  their  confidence  in  their  work.  It  would  be  extremely 
beneficial  in  future  research  to  attempt  to  objectively  document  these  reported  changes 
both  through  classroom  observation  as  well  as  test-retest  data.  This  was  not  directly 
observable  from  our  measures.  A classroom  interaction  scale,  evaluating  those  student- 
teacher  as  well  as  student-student  behaviors  one  would  expect  should  change  as  a result 
of  the  strategy  instruction,  would  provide  an  important  evaluation  component  which  has 
been  missing  from  intervention  studies  in  the  teaching  of  cognitive  strategies. 

In  further  studies  attempt  should  be  made  to  determine  differential  impact  with 
respect  to  quality  and  degree  of  program  implementation,  as  it  appeared  obvious  that 
some  teachers  were  clearly  implementing  with  greater  enthusiasm  and  commitment  than 
others.  This  might  be  built  into  the  initial  design  as  a major  factor  to  be  evaluated. 
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As  this  study  progressed  we  became  aware  of  new  instruments  which  may  in 
future  be  excellent  criterion  measures  of  program  impact.  One  such  measure  is  the 
Learning  Process  Questionnaire  (LPQ)  (Biggs,  1985).  This  particular  instrument 
evaluates  students'  learning  motives  and  strategies  in  an  interactive  way  and  thus  may 
be  a very  good  measure  of  changes  in  motivation  and  strategy  as  a result  of  program 
implementation.  The  Structure  of  the  Observed  Learning  Outcome  (SOLO)  (Biggs  & 
Collis,  1982)  method  of  determining  students'  levels  of  thinking  in  different  content 
areas  also  appears  to  warrant  consideration  as  an  approach  for  evaluating  students’ 
learning  with  respect  to  the  teaching  of  learning/thinking  strategies.  These  measures 
should  be  explored  along  with  others  in  future  studies  of  this  type. 

One  acceptable  measure  of  program  impact  is  the  degree  of  interest  generated  as 
the  project  progresses  and  the  willingness  of  other  groups  and  individuals  to  commit 
resources  and  time  to  the  program.  There  was  a significant  increase  in  interest 
exhibited  locally,  nationally  and  internationally  by  teachers,  administrators,  and 
researchers  which  has  continued  to  grow.  It  is  obvious  that  the  study  has  now 
progressed  to  the  diffusion  stage  where  the  teaching  program  (particularly  S.P.E.L.T.) 
is  now  being  incorporated  at  a variety  of  levels  in  different  classrooms,  school  systems 
and  universities. 

Programs 

The  experience  gained  from  working  intensively  with  the  two  different  programs  was 
invaluable.  The  difficulties  we  encountered  with  implementing  a program  which  takes 
time  away  from  the  content  were  many  and  varied.  One  major  difficulty  was  with  the 
conflict  teachers  felt  with  respect  to  covering  the  curriculum  in  order  not  to  detract 
from  their  students'  performance  on  the  tests  required  by  Alberta  Education.  The  felt 
need  to  cover  the  curriculum  content  was  so  great  that  in  many  instances,  it  was 
obviously  detracting  from  the  implementation  of  the  program  in  terms  of  time  allocated 
as  well  as  focus.  This  was  particularly  a problem  with  respect  to  Instrumental 
Enrichment  but  was  also  voiced  by  some  teachers  with  respect  to  S.P.E.L.T. 

Both  the  research  team  as  well  as  teachers  implementing  the  two  programs 
observed  that  it  may  well  be  that  the  principles  of  the  Instrumental  Enrichment 
program  might  be  utilized  effectively  with  some  students  displaying  particular  cognitive 
deficits  at  initial  stages  and  then  the  S.P.E.L.T.  procedures  could  be  brought  in  to  further 
extend  the  principles  developed  through  I.E.  There  might  thus  be  a blending  of  both 
programs  with  S.P.E.L.T.  assisting  extensively  in  the  bridging  and  extending  of  strategies 
to  content  areas. 
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Some  teachers  indicated  that  they  would  like  to  see  the  principles  of  teaching 
found  in  Instrumental  Enrichment  to  be  more  freely  allowed  to  be  adapted  or  extended 
beyond  the  actual  exercises  and  that  the  more  important  component  was  the  mediated 
style  of  teaching.  The  program  might  well  benefit  from  further  development  in  a 
broader  context. 

A large  percentage  of  teachers  voiced  their  concern  that  their  students  were  not 
being  tested  with  respect  to  learning/thinking  skills  but  rather  on  acquired  knowledge  of 
content  and  thus  teaching  learning/thinking  skills  had  to  be  secondary  to  teaching  factual 
knowledge  with  respect  to  curriculum  content.  If  teaching  learning/thinking  skills  is  to 
become  an  integral  and  focused  part  of  classroom  teaching,  then  assessment  of  the 
acquisition  and  application  of  learning/thinking  strategies  must  become  a part  of  the 
evaluation  of  curriculum  in  schools.  Until  it  is  obvious  that  this  is  the  case,  the  teaching 
of  learning/thinking  strategies  will  take  a backseat  to  attempting  to  cram  as  much 
factual  information  as  possible  in  order  to  answer  content-based  questions,  thus 
contributing  to  the  futile  acquisition  of  "inert  knowledge"  which  is  adequate  for  the  test 
but  fails  to  be  applied  or  evaluated  beyond  the  test  situation.  This  is  not  to  suggest  that 
teaching  content  is  not  important  but  rather  a more  prominent  role  will  need  to  be  given 
with  respect  to  teaching  the  process  of  learning/thinking. 
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Recommendations 

Based  upon  the  results  of  this  study  the  following  recommendations  are  made. 

Alberta  Education 

1 . It  is  recommended  that  Alberta  Education  make  the  teaching  of  cognitive  education 
procedures  an  integral  part  of  the  Alberta  school  curriculum  for  elementary  and 
junior  high  students  and,  that  this  become  policy.  The  vast  majority  of  experts  in 
the  field  indicate  this  should  be  integrated  into  content  teaching.  The  results  of 
this  study  would  support  this. 

2.  It  is  recommended  that  Alberta  Education  develop  and  make  available  appropriate 
resource  materials  for  teachers  to  use  in  the  teaching  of  cognitive  education 
procedures. 

3.  It  is  recommended  that  Alberta  Education  develop  and  make  available  to  teachers 
and  school  administrators  cognitive  strategy  assessment  instruments  in  differing 
content  areas  as  well  as  affective  domains.  These  should  be  appropriate  for  both 
formative  and  summative  evaluation  purposes.  This  assessment  might  be  made  as 
part  of  Alberta  Education's  ongoing  program  of  testing  for  school  jurisdictions. 

4.  It  is  recommended  that  Alberta  Education  make  available  to  all  its  field  consultants 
training  opportunities  in  theory,  research,  and  application  issues  relating  to 
cognitive  education  procedures. 

Universities 

1 . It  is  recommended  that  teacher  training  programs  in  Alberta  provide  compulsory 
training  in  the  principles  and  practice  of  cognitive  education  for  all  preservice 
teachers. 

School  jurisdictions 

1 . It  is  recommended  that  inservice  training  be  made  available  with  respect  to 
methodologies  for  teaching  and  evaluating  learning/thinking  strategies  in 
classrooms. 
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Joint  university,  Alberta  Education  and  school  jurisdictions 

1 . It  is  recommended  that  a Cognitive  Science  Unit  be  established  which  would  have 
the  following  as  its  major  functions: 

( i ) the  inservice  training  of  teachers; 

( i i ) assisting  in  the  development  and  implementation  of  preservice  training 
courses  at  the  university  level; 

( i i i ) providing  consultation  to  school  systems  regarding  implementation  and 
evaluation  procedures; 

( i v ) conducting  basic  and  field  research  on  the  teaching  and  assessment  of 
cognitive  education  procedures. 

The  setting  up  of  this  unit  should  initially  be  a joint  endeavor  between  Alberta 
Universities,  Alberta  Education,  and  school  jurisdictions. 

2.  It  is  recommended  that  additional  research  be  conducted  regarding  the  efficacy  of 
cognitive  education  procedures  at  primary  and  secondary  levels  as  well  as  with 
different  populations,  particularly  the  mildly  retarded  and  native  students.  There 
is  at  present  little  systematic  research  on  the  effectiveness  of  these  approaches  for 
these  students  in  mainstream  classrooms. 

Concluding  Remarks 

Education  has  tended  to  easily  grab  on  to  new  instructional  approaches,  the  most  recent 
one  being  learning/thinking  strategy  teaching  or  in  somewhat  more  popular  terms 
"metacognitive  instruction".  This  type  of  instruction,  regardless  of  which  particular 
program  is  used,  attempts  to  teach  students  to  plan,  implement,  monitor  and  evaluate 
specific  strategic  approaches  to  tasks.  The  recent  literature,  including  this  study, 
suggests  that  this  type  of  instruction  has  the  potential  to  make  enduring  positive  changes 
with  respect  to  student  learning  and  problem  solving  (see  for  instance  Haller,  Child,  & 
Walberg,  1988;  Paris,  & Oka,  1986,  1989;  and  Palincsar,  & Brown,  1987).  It  is 
likely  that  no  one  particular,  added-on  program,  at  one  instructional  level,  will  provide 
the  adequate  emphasis  needed.  In  fact,  some  experts  in  the  field  suggest  that  it  may  not 
be  what  you  teach  (in  terms  of  particular  strategies  or  materials  used)  but  how  you 
teach  it  that  is  most  critical  to  positive  student  change  (Nickerson,  1988).  Many 
teachers  in  the  study  reported  here  would  also  support  this  view. 

The  results  of  this  study  are  very  encouraging.  There  are  potential  benefits  to 
students  and  teachers  of  implementing  cognitive  education  procedures  in  mainstream 
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elementary  and  junior  high  school  classrooms.  The  study  reported  here  suggests  that  a 
number  of  positive  changes  in  student  behaviors  do  occur  for  different  types  of  learners. 
The  approaches  examined  in  this  study,  however,  are  only  two  out  of  a wide  variety 
available  and  these  two  might  best  be  viewed  as  initial  attempts  at  teaching  learning/ 
thinking  which  do  hold  some  promise.  Further  development  and  evaluation  is  still 
necessary. 

There  are  numerous  questions  still  to  be  answered  within  the  context  of  this 
particular  study  including  developmental  and  individual  differences  of  the  different 
groups  of  students  with  respect  to  strategy  and  affect.  As  well,  questions  regarding  more 
specific  subgroup  differences  with  respect  to  program  impact  are  still  left  to  be 
answered  (e.g.,  those  high  in  self-concept  versus  low,  those  high  in  external  locus  of 
control  versus  low,  etc.).  As  well,  examining  subgroups  of  students  who  appeared  to 
benefit  to  a greater  degree  than  others  and  then  attempting  to  determine  why  this  is  the 
case  would  provide  additional  insight  into  differential  effects  of  program  impact.  These 
and  other  analyses  would  help  to  provide  a more  comprehensive  picture  leading  to  a 
better  understanding  of  the  specifics  involved  in  this  type  of  intervention.  There  are  also 
many  questions  yet  to  be  answered  in  future  work  including:  How  quickly  should  one 
introduce  a new  strategy?  What  is  the  most  powerful  way  to  obtain  transfer  and 
generalization?  What  is  the  best  way  to  interface  content  and  strategy?  What  is  the 
most  appropriate  way  to  integrate  affect  and  cognitive  strategy  teaching? 

Despite  the  problems  and  unanswered  questions  we  need  to  pursue  metacognitive 
instruction  in  our  classrooms.  There  is  clearly  the  need  to  provide  for  a comprehensive 
integrated  approach  to  the  teaching  of  cognitive  strategies  across  all  levels  of  education 
beginning  at  kindergarten  through  to  post-secondary.  The  most  appropriate  ways  of 
doing  this  to  enhance  learning/thinking  have  yet  to  be  determined.  However,  we  do  now 
know  enough  to  begin  to  make  a start. 
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